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COMPUTER PROGRAM TO MINIMIZE PREDICTION ERRORS IN MODELS FROM EXPERIMENTS 


WITH 16 HYPERCUBE POINTS AND ZERO TO SIX CENTER POINTS 

Arthur G. Holms 

National Aeronautics and Space Administration 
Lewis Research Center 
Cleveland, Ohio 44135 

SUMMARY 

An important and efficient experiment is provided by the full or frac- 
tional two-level factorial experiment with 16 factorial points and zero to 
six center points. For such experiments, a previous report described a back 
ward deletion procedure of model selection that was optimized for minimum 
prediction error and which used a multiparameter combination of the F-dis- 
tribution and an order statistics distribution of Cochran's. A computer 
program is now described that applies the previously optimized procedure to 
real data. The use of the program is illustrated by examples. 

Prior art was defined as using the F-distribution with backward dele- 
tion, testing terms in the nondecreasing order of their absolute magnitudes. 
Such a procedure is here called an F-only strategy. Typically, backward de- 
letion has been used with a nominal significance level chosen to control the 
probability of Type 1 errors at a nominal level of 5 percent. The 5-percent 
level F-only strategy produced enormous increases in prediction errors over 
the recommended (multiparameter) strategies. A 50-percent level F-only 
strategy still produced small increases in prediction errors over the recom- 
mended strategies. 


INTRODUCTION 

Whereas experiments that elucidate the mechanism of a process should 
usually be preferred over experiments that merely give empirical data, exper 
iments that result in empirical mathematical models often provide the quick- 
est way to optimize a process, a material composition, or a structure. 
Techniques sometimes called Box-Wilson methods or, more recently, response 



surface methods have been found to so systematize empirical investigations 
that an efficient approach to optimum conditions is assured. 

In materials and structures development, the experiments are often char- 
acterized by 

(1) Costly experimental units such as cast or forged specimens. 

(2) Time-consuming techniques such as fatigue or rupture testing. 

(3) Error-accumulating processes such as high-temperature melting or 
forging and testing in corrosive environments. 

The two-level, fixed-effects, full- or fractional-factorial designs of 
experiment, without replication, are often appropriate for these situations. 
Some examples of work that fall within the preceding framework are described 
in Collins, Quigg, and Dreshfield (1968); Sandrock and Holms (1969); Kent 
(1972); and Filippi (1974). 

Subset regression procedures are often used to select the terms that 
are retained in a multivariable polynomial equation that has been fitted to 
the results of an experiment. The selection or rejection of terms is usually 
done according to a statistical test at some arbitrary level of significance. 
The usual purpose of the final model is to predict the value of the depen- 
dent variable as a function of values of the independent variables that may 
be of interest. For such a purpose, the selection of terms should be done 
to minimize prediction error rather than to achieve an arbitrary level of 
significance. 

The purpose of this report is to describe and illustrate the use of a 
previously optimized method (Holms (1980)) fo; the selection of multivariable 
polynomial models to be fitted to the results of two-level factorial experi- 
ments having zero to six center points. (The method was optimized to mini- 
mize the maximum prediction error over the points of the experiment.) A 
computer program that uses the method and its application to real experiments 
is i 1 lustrated by two examples. The context of the development of the method 
and its justification are outlined in appendix A. 
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SEQUENTIAL DELETION 


Population Model 

The single observation value of the response is assumed to occur accord- 
ing to the model 

y = E(Y) + e (1) 

where the error e is independently normally distributed with mean zero and 
variance a . (Some robustness against nonhormality for a chain-pooling 
procedure was demonstrated by Holms and Berrettoni (1967).) 

For relatively saturated experiments smaller than 16 observations, the 
opinion is offered that such experiments are too small to provide both good 
estimates of model coefficients and a good test statistic in cases where 
random errors are large enough to call for a statistical decision procedure. 
The simulations of Holms (1980) were all performed with experiments contain- 
ing 16 hypercube points plus zero to six center points in the belief that 
such experiments are large enough to justify the use of a statistical deci- 
sion procedure but small enough so that the precise optimization of the pro- 
cedure would be quite beneficial. Where g is the number of independent 
variables and the experiment is assumed to be a 2”*^ fractional replicate 
of the full-factorial experiment, factorial observations are assumed to re- 
sult one-for-one from the hypercube points and their number is 

= 16 
c 

An example of an appropriate model equation for the population mean 
value of the response in the case of four independent variables is 



Model Fitting 


The model is assumed linear with orthogonality of parameter estimates 
provided by the design of the experiment. An equation such as (2) is assumed 
fitted to the results of a two-level experiment where the x's are design 
values, namely, the high level of X|^ is represented by X|^ = +1, and the 
low level of Xj^ is represented by X|^ = -1. (If center points are used, 
they have coordinates with all Xj^ = 0.) The initial model fitting is as- 
sumed to give least-squares estimates of the model parameters that are mini- 
mum variance unbiassed linear estimates, and for parameters beyond the 
estimates have the form 

"c 

-irS ’ikJ'k (3) 

k.i 

where i = 2, n^ and the a^|^ are appropriate values of ±1. Such 
estimates have population mean values 

E(b^) = Bi (4) 


Combination Estimate for Zero Degree Coefficient 

A weighted estimate of the Bj of equation (2) is to be formed from 
the n^ hypercube observations and the Oq center point observations where 
all observations are assumed to have variance a^. Model coefficients esti- 
mated from the hypercube observations each have variance 

V(b.) = a^/n^. 

Thus, the variance of the function estimate for a model such as equation (2) 
with coefficients all estimated, for example, by Yates' method, from such ob- 
servations is (at the design center) 

V(Yq) = V(b^) = a^/n^ 
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Let Yq be estimated from a combination of the Oq hypercube points and 
the n^ center points. Let be the arithmetic mean of the n^ center 
point observations. Then the estimate of Yq weighted inversely as the 
variances of b^^ and Yq is 

^0 ' * Vo>'("c * "o> 

Because the coefficient estimates are uncorrelated, the weighted estimate Yq 
is also the least-squares estimate of Thus, if b^ is the estimate of 

the zero degree coefficient from the Yates analysis, the least-squares esti- 
mate from the combined observations is 


bi = (n^bj + n(,y„)/(n^ + ng) 


(5) 


Mean Squares and Sums of Squares 


The squares of the estimates multiplied by n^ provide the numerator 


mean squares used in the hypothesis testing: 


z. = n bt 
1 c 1 


( 6 ) 


These mean squares have expectations 


E(Z,) - 


( 7 ) 


where 


V(YJ . o‘ 


( 8 ) 


for i = 2, ..., n^ and k = 1, ... , n^. Thus, from equation (7), if any 

2 

IS zero, the associated Z^- is an estimator of a . 


Hypothesis Testing 


The denominator for the hypothesis testing is based on the construction 
of sums of squares. Six cases are identified according to combinations of 
the values of Oq and mp, where ng is determined by the design of the 
experiment and m is chosen according to a strategy of hypothesis testing. 
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The cases are identified by the first three columns of table 1(a). Equations 
for the initial sum of squares to be used in the starting denominator of the 
test statistics are derived in appendix B and given in the fifth column of 
table 1(a) . 

The following remarks apply to the cases listed in table 1(a). 

Because case a provides no denominator sum of squares, there can be no 
deletion procedure. All the model coefficients are estimated and all the 
terms are retained. 

Case d uses Oq = 0 and mp > 0. This is the case investigated by 
Holms and Berrettoni (1969). The deletion method of that investigation was 
as follows. 

The mean squares Z^. from the usual Yates analysis (aside from the 
zero degree coefficient) are ordered in nondecreasing magnitude and renamed 

Z(j)^ 

Z(i) < ••• < 


where j = 1, . 
the smallest Z 


n and 


n = n^ - 1. 
c 


Chain-pooling assumes that m of 


have been generated with zero population coefficients. 
Their sum is called where initially 



Multiplication of the critical points of Cochran's (1941) distribution by j 
gives the critical points of the U. -distribution tabulated by Holms and 

J 

Berrettoni (1967). The mean square is tested for significance at nom- 

inal level a using the statistic 



(9) 


If Z/ -X is not significant, j is indexed upward by one and the next mean 

t h 

square is tested. If the k -ordered mean square is the smallest mean 
square to test significant at level a, then all terms associated with smal- 
ler mean squares are deleted from the model. 

Consider cases b and c where > 0 and mp = 0. The first mean 
square to be tested is the null hypothesis is 
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Hq: - 0 


( 10 ) 


where, for any j, is the parameter associated with the ordered mean 

square The alternative hypothesis is 

^a‘ ^(1) > ° 

Because Uj is not defined for j < 2, cannot be tested against 

the U .-distribution. If the test is performed against the F-distribution, 

vJ 

the fact that is an ordered statistic implies that a test of nominal 

size a will not have true size a. With this proviso, a nominal size 
a-test is performed. 

For case b, the initial test statistic is 


"dfb^(l) 

l,ndfb ■ 


( 11 ) 


with n^^j^ defined by equation (B3) and computed by equation (B4). 

For case c , the equivalent initial test statistic is 


_ ^dfc^(l) 
l,ndfc SS 

c 


( 12 ) 


where n^^^ is given by equation (B6) and SS^ is given by equation (B5). 
If Z^j is reported significant, no further testing is done and there is no 
conditional deletion of terms. 

For either case b or c , let SSq be the initial sum of squares and 
n^fO the initial degrees of freedom. If Z^^ was reported as insignifi- 
cant, then it is pooled with SSq for a test of ^^ 2 )’ statistic 

is then 


^ Vo ^^^(2) 

l,ndfO+l “ SSq + 


( 13 ) 


Testing and pooling continue in this manner provided that insignificance is 
the result of the prior test. 

The test statistic for any is thus 
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For j > 1, the option exists of testing against the F-distribu- 

tion or against the Uj-distribution. These options are also both available 
for the first test of a in cases d, e, and f; however, testing against 

the F-distribution might not be good for case d, because in case d there is 
neither a pure error nor a residual sum of squares, and the testing is per- 
formed entirely with ordered mean squares. 

Suppose the situation is. that of > 0 and j > 1. A criterion is 
needed for choosing between testing against the F-distribution or the U— dis- 

J 

tribution. If j is relatively small and ng is relatively large, the 
F-distribution might be more appropriate; whereas, if j is relatively large 
and ng is relatively small, the Uj-distribution might be more appropriate. 
The approach was for j > 1 to compute j/ng and use the F-distribution for 

- *^F Uj-distribution for j/ng > rp, where 0 < rp < “ and 

rp was optimized from Monte Carlo studies. Table 1(b) shows how the choice 
of the value of rp affects the values of ng and j at which a transfer 
occurs from the use of the F-distribution to the use of the U .-distribution. 

J 

Consider the case 

j/ng > rp (15) 


The Uj statistic is defined by equation (9). Suppose the criterion 
j > rpOg has been met and the information in yg^ ..., yg^g is to be 
combined with that in 


, ..., Z/. X for a test of Z/ against a 


critical point of the U --distribution. An approximation to equation (9) is 

vJ 


, <Vo 

j ■ * ... . z,j, 


The distribution of t. of equation (16) is merely an approximation of 

J 

the distribution of Uj of equation (9) because the denominator in equation 
(16) has been stabilized by the n^pg mean squares in SSg. 

Under the null hypothesis, z^j^ is the largest of an ordered sample of 
j estimators of . Although the quantity (SSg + z^^ + ... + 2^^^)/ 

(Odfo + j) is an estimator of as is the quantity (>^j_]^ ^(j)^^"^ 


8 



equation (9), the quantity j of equation (16) rather than the quantity 

(ndfo + j) was used as the entry point of the Uj-tables because the tables 

are based in part on the numerator Z/ being the extreme value of a 

V J / n 

sample of mean squares together having a mean value of 0 . 

For case d , with n^. = 0, no testing or deletion can be done unless 


'0 

mp > 0. The initial value of j is 
m 


th 


I + 1. Thus, even if «p < 1.0, wi 
> 0, j is > 1, and with n^ = 0, j is > rpOg, and testing against the 


F-di stribution is excluded. Thus, any testing with ng = 0 is done 
against the Uj-distribution, 
equation (9) takes place with 


against the U —distribution, which implies that the first test using 

\J 


"■j-l - SSd 


where SS^ is defined by equation (B7) and j = rtip + 1. Subsequent testing 
is done with 




" 


+ Z 


(j) 


For either case e or f , with j < rpOQ, the test statistic is provided 
by equation (14) with SSq and being given by equations (B9) and 

(BIO) for case e and by (Bll) and (B12) for case f. 


Terms Deleted 

If tests as insignificant, it is pooled with (added to) the de- 

nominator of the test statistic and j is indexed upward one unit. When a 
tests as significant, testing is stopped and n terms are deleted from 
the model where n is the integer value of r^(j - 1) and r^ has 
been empirically optimized. If Z^j^ tests as insignificant at j = n^ - 1, 
then n is the integer value of - !)• The terms deleted are those 

corresponding to the 0 smallest mean squares. 
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Definition of Strategy 


In summary, the expressions for degrees of freedom and sums of squares 
are given in table 1(a), and the entire sequential deletion strategy is spe- 
cified by the parameters (mp, rp, ap, ay, r ). Where 0 < m < n , m is the 

^ — p c p 

number of mean squares initially pooled. If is the ordered mean 

square being tested, then is tested against the Uj- distribution at 

nominal level oy if j > I and j > npng. The mean square Z^j^ is 
otherwise tested against the F-di$tribution at nominal level op. The con- 
vention ap = 1.0 is used to signify that no testing is done against the 
F-distribution, and the convention of ay = 1.0 is used to signify that no 
testing is done against the 1) —distribution. The number of terms found to be 

o 

insignificant is multiplied by r^ and the integer value of the product 

(namely, n) is the number of smallest absolute value coefficients whose 

* 

terms will be deleted from the model. (The coefficient bj, of the zero 
degree term is excluded from the deletion procedure.) In the notation for 
the strategy parameter set (m^, rp, ap, ay, r^), a long dash is used to rep- 
resent a parameter if it has been made inoperative by a value assigned to 
some other parameter. 

Selection of Strategy Parameters 

Some of the more useful strategies investigated by Holms (1980) are sum- 
marized in table 1(c). The type a strategy should be used when the error is 
known to be small (coefficient of variation in the vicinity of 4.0 percent). 
The type b strategy should be used when the relative magnitude of the error 
is unknown. The type c strategy should be used where the relative error is 
known to be large (coefficient of variation as large as 65 percent). 

COMPARISON WITH MORE USUAL METHODS 

How do these multiple parameter strategies compare with the usual 
strategy? An attempt to answer the question requires some assumptions as to 
what constitutes a usual strategy. Commonly, a subset regression procedure 


10 


is used with coefficient deletion at an arbitrarily defined level of signifi- 
cance. As developed in appendix A, very little work has been done to iden- 
tify procedures that minimize prediction error. The following assumptions 
defining a usual strategy are now introduced: 

(1) The deletion procedure is based exclusively on the critical points 
of the F-distribution. 

(2) The procedure uses stepwise sequential deletion, testing terms in 
the nondecreasing order of their absolute magnitudes. 

(3) There is no arbitrary reintroduction of insignificant terms as 
implied by values of r^ < 1.0. 

(4) There is no arbritrary initial deletion of terms as implied by 
values of mp > 0. This assumption requires tiq > 0. 

The preceding assumptions imply that a usual strategy (henceforth, 
"F-only" strategy) can be compared with the proposed strategies by defining 
the F-only strategy as a strategy where (mp, rp, «p, a^, r^) have the values 
(0, 16.0, Op, 1.00, 1.0) and where op is chosen according to some criterion 
satisfactory to the analyst. 

Results for the F-only strategies are given in table 1(d) and are com- 
pared with the multiparameter strategies in figure 1. The curves (fig. 1) 
for the F-only strategies always lie above the curves for the multi parameter 
strategies; that is, the F-only strategies are everywhere dominated by the 
multiparameter strategies and the F-only strategies are not admissible. 

In addition to the coefficient of error functions C shown in 

table 1(d), regret functions R(v) (defined in appendix A) are also listed. 
For values of tiq from 1 to 6, the value of a that jointly minimizes both 
R(64.3) and R(4.0) is here called a security F-only or optimized F-only 
strategy. Table 1(d) shows that for all values of tiq from 1 to 6 the opti- 
mizing value of a is 0.50. This result suggests that, whereas a = 0.05 
is a very popular value for significance testing, a value of 0.50 might often 
be preferred if the object is to minimize prediction error. 

The values of C F-only strategy with o = 0.05 and for 

d 0 ^ InX 

the F-only strategy with a = 0.50 are compared with the values of C,. ^„(v) 
for the security regret multi parameter strategy in table 1(e). For the more 
usual F-only strategy where a = 0.05, the increase in prediction error over 
the range of nQ was in the range of 19.2 to 28.9 percent at v = 64.3 and in 
the range of 157.3 to 901.8 percent at v = 4.0. When the F-only strategy 
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was used with the optimum a (namely, 0.50), the increase in prediction 
error varied from 4.7 to 8.8 percent at v = 64.3 (large coefficient of 
variation), and the change in prediction error varied from a decrease of 1.5 
percent to an increase of 1.1 percent from the multiparameter values at 
V = 4.0 (small coefficient of variation). 

USE OF COMPUTER PROGRAM 

Application of the computer program (POOLIO) is illustrated herein by 
two examples. A detailed description of the program input and output is 
given following the description of the second example. A description of the 
operating sections is given in appendix C, and the program listing is given 
in appendix D. 

5 

Illustrative Example: One-Half Replicate of 2 Experiment 

Without Center Points 

The alloy development experiment that produced the data was described by 
Sandrock and Holms (1969). The levels of composition and pour temperature 
data are given in table 2(a) in which the levels of the independent variables 
are stated in natural units. The compositions melted are given in design 
units in table 2(b), and the mechanical property data are given in natural 
units. 

One casting was made for each of the 16 compositions listed in table 
2(b). Two stress-rupture tests from each composition were run at 1010° C and 
103 MN/m^ (1850° F and 15 ksi). The results are also shown in table 2(b). 
Testing variability and variability within castings could be observed from 
the test bar duplicates, but since there were no melt-to-melt duplicates, 
melt-to-melt variability could not be observed. A later investigation of 
melt-to-melt variability showed that the melt-to-melt variability signifi- 
cantly exceeded the bar-to-bar variability. 

The next step is to subject the data to the model-fitting and deletion 
procedure of the computer program POOLIO. The initial step is to use the 
logarithmic transformation to transform the values of the response t 
(stress-rupture life in hours) to logarithms of stress-rupture life y in 
order to generate a variable whose variance (scatter) would be approximately 
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constant over large changes in stress-rupture life that might result from the 
changes in composition. 

The program reads out the input data and the mean value of the paired 
logarithms as illustrated by the values of Y(l), Y(2), and Y(I), respec- 
tively, in the illustrative output in appendix E. The model coefficients 
were then fitted by Yates' method, and sequential deletion procedures were 
invoked using the strategy parameters for = 0 listed in table 1(c). 

(The Yates method of the computer program attaches algebraic signs to the 
estimated coefficients as if the experiment were a full-factorial experiment 
on four independent variables. For the half-replicate experiment that was 
actually run, the signs that must be reversed are shown by the negative signs 
attached to the coefficients in table 7 of Holms (1967).) 

Results are listed in table 3. The security regret strategy suggests 


that all terms of the model be retained; however, the three largest absolute 
value coefficients are all coefficients of first-degree terms. The strategy 
for large error suggests that two interaction terms and Xj^Xj be 

retained; however, they have coefficients that are clearly smaller in abso- 


lute \/alue than the coefficients of 


"Cr’ "Al’ 


x^. Furthermore, the 
• are not much smaller 


absolute values of the coefficients of Xy and Xj^ are not much smaller 
than the absolute value of the coefficient of Xj^Xj. Thus, the first-degree 
equation of the method of steepest ascents is a reasonable approximation to 


the data; namely. 


log t = 1.652 - 0.383 Xqj„ - 0.146x^1 + O.lOOXj^ 

+ 0.037Xj - 0.030Xj^. 

4 

Illustrative Example: Full Replicate of 2 Experiment 

With Center Points 


As described by Sandrock and Holms (1969), a vector experiment was de- 
signed and performed based on the results shown by the preceding equation. 
The results in the preceding equation and the results of the vector experi- 
ment were then used to design a second factorial experiment. It was per- 
formed without center point replication, and the conditions and results are 
shown in table 4. 
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The purpose of this section is to illustrate the use of the computer 
program POOLIO when center point data are available; however, no center point 
data were acquired until a star block (table V of Sandrock and Holms (1969)) 
had been performed. To use those center point data in this discussion, a 
mean value of the stress-rupture times t^ was computed for the observations 
of the hypercube experiment, and a mean value tg was computed for the center 
point observations. The difference was used as an estimate of the block dif- 
ference b^^ between the star block and the hypercube block as follows: 


= 228.4 - 144.3 
= 84.1 

The value 84.1 was then subtracted from the actual center point observations 
to obtain the adjusted center point observations used in the present example. 
The data from table V of Sandrock and Holms (1969) so adjusted are as follows: 


— 

Alloy 

Stress-rupture 

life, 

hr 

C-1 

195.2, 

185.6 

C-2 


88.0 

C-3 

149.5, 

119.8 

C-4 

158.7, 

143.2 


Without the use of center points, the coefficient estimates using the 
computer program POOLIO are as follows: 


Term 

Coefficient estimate 

Term 

Coefficient estimate 

xO 

2.139 

XAl 

-0.063 

XTi 


^TiXAl 

-.054 

^Cr 


^Cr^AI 

-.021 

XT iXQy. 


XTi^Cr^Al 

.025 



XCXAI 

.077 

^Ti^C 


XTi^C^Al 

.031 

xcr^C 

-.011 

xCr^C^Al 

-.025 

m^CrXC 

-.026 

XTiXCr^C^Al 

.013 
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Because the coefficient of uses center point data in its estimate, 
its value changes with nQ. The values are as follows: 


"^0 

Coefficient of x^ 

0 

2.139 

1 

2.148 

2 

2.140 

3 

2.139 

4 

2.141 


Because the two-level full- or fractional-factorial experiment provides 
orthogonal coefficient estimates, the estimates of coefficients that remain 
are not affected by the deletion of coefficients. The number of coefficients 
beyond the coefficient of x® is given as p by the deletion strategy, 
which means that the p largest absolute value coefficients and associated 
terms are to be retained in the model. The values of p for the different 
strategies and for differing possible numbers of center points are given in 
table 4(c) . 

As listed in table 4(c), the value of p for the security regret 
strategy for the largest value of Oq was p = 6. In the absence of any 
other information, this value of p should be accepted as giving the model 
equation with minimum prediction error. Using the coefficient of x^ for 
n^ = 4 with the other six largest absolute value model coefficients results 
in a predictive model as follows: 

logout = 2.141 + O.OSIxq - 0.026x-|.^Xj,^Xq 

- 0.063x^1 - 0.054x^^x^1 + 0.077x^Xy^1 
+ 0.031x^^x^x^^ 

The form of the preceding equation shows very clearly that a first- 
degree equation cannot be fitted to the data and that the use of a vector 
experiment in the method of steepest ascents would be inappropriate. The 
next step should be the performance of a star block so that a quadratic model 
equation can be obtained for use with the method of local exploration, as was 
done by Sandrock and Holms (1969). 
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Description of Computer Program Input and Output 

POOLIO is a FORTRAN-4 program that uses real data as input and performs 
the backward deletion using the statistical procedures developed in Holms 
(1980). The program is listed in appendix D, illustrative input is shown in 
figure 2, and illustrative output is given in appendix E. The following 
description of input and output has some paragraphs labeled identically with 
the applicable program sections in appendix D. 

Section lA - Declarations and tables . - The values of the nominal test 
size a are stored as (ALPHA(I), I = 1,11) and later used as output 
labels. These values range from 0.001 to 1.0; however, the value of 1.0 
obtained by setting the index to 11 is merely a code implying that no signi- 
ficance testing is performed. 

The sequential deletion requires critical values against which the test 
statistics are compared. The critical values of F are stored internally 
as ((FTB(I,J), J = 1,10), I = 1,20) where I indexes on the degrees of 
freedom and J indexes on the value assigned to a. The critical values 
of U. are stored internally as ((TB(I,J), J = 1,10), I = 1,16) where I 

J 

is the order number in nondecreasing order and J indexes on the value as- 
signed to a. 

Section IB - Inputs and constants. - A logarithmic transformation of 
the dependent variates is provided. If KODE > 1, the logarithmic transfor- 
mations are performed at the statement preceding statement 102. 

Two-level, full or fractional experiments are sometimes performed in a 
manner that provides superficial rather than true replication. For example, 
if the treatments are each applied to different specimens but the dependent 
variable is observed through duplicate measurements on each of the specimens, 
such replication does not measure the specimen-to-specimen variability. A 
conservative procedure would be to compute the means of the repeated mea- 
surements and then proceed with these mean values as if they were observa- 
tions from an experiment with no replication. The program computes the mean 
values in such cases of superficial replication for NYIN variates in each 
data set corresponding to a unique treatment. The operation is completed at 
statement 103. 

The constants defining the input data and the sequential deletion 
strategy are read from data cards in the following order, with the order of 
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the fields being the same as the order of the symbols in the following des- 
cription: 


Format 


Description 


(13A6,A2) 


( 211 ) 


(A6, 

16, 


6F6.0) 

(3I5,2F5.3) 


REMARK (I) 


NO, KODE 


DATAID 

NYIN 


YIN 

MP, KPF, KPU, 
RF, RETA 


Arbitrary literal information such as par- 
ticular use of program and date of last 
change. 

NO is number of true center point replica- 
tions. If KODE < 1, dependent variates 
are not transformed. If KODE > 1, loga- 
rithmic transformation is used. 

Treatment identification (may be omitted). 
Amount of superficial replication. Arith- 
metic mean of this number of variates is 
computed after any transformation and before 
model fitting. 

Response variates. 

(The associated READ statement is actually 
in section 3.) 


Illustrative input data are shown in figure 2. If the data are punched 
into data cards, the input is as follows. The first card gives the literal 
information in (13A6,A2) format stating that the computer run uses data from 
a particular experiment (namely that of table 4) and that the data were 
keypunched on 04-02-82. The second card in (211) format states that NO, 
the number of center point observations, is 4 and that KODE = 1, which im- 
plements the logarithmic transformation of the dependent variates. The next 
16 cards in (A6, 16, 6F6.0) format give the basic information of the hyper- 
cube points. The first six columns in A6 format were used to reproduce the 
alloy designation of table 4(b). The next six columns in 16 format were 
used to record the number of superficial replicates NYIN, which in all cases 
was 2. Columns 13 through 48 are available in 6F6.0 format and in the 
present instance columns 13 through 18 were used to record the observation of 
126.7 hours for alloy 1 of table 4(b), and columns 19 through 24 were used to 
record the observation of 176.5 hours for alloy 1 of table 4(b). Of course. 
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all 16 cards for the alloys numbered 1 through 16 are hypercube points and 
must be in the proper order for the operation of Yates' method of coefficient 
estimation. Following the information for the 16 hypercube observations are 
the cards containing the center point information in the same (A6, 16, 6F6.0) 
format as was used for the hypercube data. The number of such cards must be 
the same as was specified previously by NO. 

Following the center point data cards are the strategy specifications in 
(315, 2F5.3) format where the value of MP is given in the first five col- 
umns, the value of KPF is given in columns 6 to 10, the value of KPU is 
given in columns 11 to 15, the value of RF (namely, rp) is given in columns 
16 to 20, and the value of RETA (namely, r^) is given in columns 21 to 25. 
The relation between KPU or KPF and ay or op is as follows: 


KPU or KPF 

a(j or ap 

1 

0.001 

2 

.002 

3 

.005 

4 

.01 

5 

.025 

6 

.05 

7 

.10 

8 

.25 

9 

.50 

10 

.75 

11 

1.00 


Thus, the last three cards of figure 2 specify strategies as follows: 


mp 

TF 

«F 

«u 

'^n 

0 

0.5 

0.75 

0.50 

0.85 

0 

.5 

.50 

.50 

.80 

0 

.9 

.10 

.05 

.80 


Section 5 - Output . - On completion of the deletion of terms, the final 
step is writing the input and output data. Such data are illustrated in 
appendix E for the input data of figure 2. The first line of output merely 
repeats the literal information on the first data card. The next line gives 
the values of n^, n^, and KODE by means of the statement 

NC = 16 NO = 4 KODE =1 
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(KODE = 1 shows that a logarithmic transformation is used). Following that 
statement are column headings for the input data, namely, the literal infor- 
mation identifying the observation, followed by the observations, where there 
might be as many as six superficial replicates as indicated by the column 
headings Y(l), ..., Y(6). The input hypercube and center point observations 
are then listed in E 13.5 format under those column headings. Following 
that listing is the two-column array headed DATAID and Y(I), and under 
these column headings are the original literal data identifications and the 
mean value of the transformed observations (under Y(I)). 

The statements NDF = 3 and S = 0.66830-01 give the values of n^^ 
and Sj in 13 and E 13.5 formats as computed prior to any sequential de- 
letion. The next line of output gives the values of the strategy parameters 
(nip, r^, ttp, oy, r^) which, for the present example, are given by the line 

MP =0 RF = 0.500 ALPHAF = 0.500 ALPHAU = 0.500 RETA = 0.800 

Values of p, n, and the smallest mean square concluded to be significant 
are given by the line 

IRHO = 6 JETA =9 Z(JETA + 1) = 0.11194-01 

The column headings DATAID and B(l) indicate that the columns con- 
tain the original data identifications together with the coefficient esti- 
mates (however the coefficients might be aliased) in the order they are com- 
puted by Yates' method. These estimates reflect the consequences of the de- 
letion procedure in that deleted coefficients have been set equal to .00000. 

The last line gives the values of n^^ and s^ as estimated subsequent 
to the deletion procedure by the line 

NDF =15 S = 0.23759-01 

The last three lines of figure 2 show three sets of strategy parameters 
(fflp, rp, Op, ap|, r^) as input data. For every such input card there is a 
separate output line with the symbols 
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MP = 


RF = 


ALPHAF = 


ALPHAU = 


RETA = 


and all the subsequent output as just described. 

CONCLUDING REMARKS 

A computer program was presented that combines the familiar Yates 
method of model coefficient estimation with a sequential backward deletion 
procedure. In contrast with many subset regression procedures, the deletion 
procedure does not attempt to reject terms according to some arbitrary sig- 
nificance level but, instead, is based on model-deletion strategies that were 
optimized for minimum prediction error. The optimization was done for two 
level factorial experiments containing 16 factorial points and any number 
from zero to six center points. As such, the procedure was optimized for 
situations containing very meager pure replication, and provision was made 
for the conditional use of insignificant coefficient estimates to improve 
the stability of the test statistic. 

An attempt was made to define what might be called a prior art of se- 
quential deletion for comparison with the recommended strategies. The prior 
art was assumed to use the F-distribution with backward deletion, testing 
terms in the nondecreasing order of their absolute magnitudes. Such a proce- 
dure is here called an F-only strategy. Typically, backward deletion has 
been used with a significance level chosen to control the probability of 
Type 1 errors at a nominal level of 5 percent. The present investigation 
showed that a good F-only strategy for minimizing prediction errors would 
use a nominal significance level of 50 percent. 

The 5-percent level F-only strategy produced enormous increases in pre- 
diction errors over the recommended (multi parameter) strategies. The 50-per- 
cent level F-only strategy still produced small increases in prediction 
errors compared with the multiparameter strategies. In particular: 

1. The 5-percent level F-only strategy increased prediction errors 20 
to 30 percent at a large coefficient of variation and 150 to 900 percent at 
a small coefficient of variation over the prediction errors for the multi- 
parameter strategies. 
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2. The 50-percent level F-only strategy (optimal among F-only strate- 
gies but seldom used) still increased prediction errors 5 to 9 percent at a 
large coefficient of variation but resulted in essentially no change at a 
small coefficient of variation over the prediction errors for the multipara- 
meter strategies. 
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APPENDIX A 


DEVELOPMENT OF DELETION PROCEDURE 
Background 

For two-level, fixed-effects factorial experiments without replication 
a suitable practice consists, according to Davies (1956), of pooling some 
arbitrary number of the highest order interaction mean squares into an esti- 
mate of error variance. When this is done, any of these high-order inter- 
actions that are not actually small or any unknown block effects (major 
changes in experimental conditions not accounted for by the model) could 
inflate some of the pooled interactions and, thereby, give too large an esti- 
mate of error variance. Too large an error estimate reduces the sensitivity 
of subsequent tests. 

The preservation of sensitivity, when pooling mean squares into the 
estimate of error variance, has been an object of the procedure of Daniel 
(1959) and of Wilk, Gnanadesikan, and Freeny (1963). 

Daniel (1959) uses the absolute values of the effect estimates as order 
statistics. They are plotted on probability paper, and the result is called 
a half-normal plot. In addition to conditional structuring of the model, 
Daniel's objectives included the determination of "bad values, heteroscedas- 
ticity, dependence of variance on mean, and some types of defective randomi- 
zation, ..." The half-normal plot combined with a background of experience 
might provide a method by which a skillful user could pass judgment on the 
results of an experiment. Daniel concluded that such a plot can be used to 
make judgments about the reality of the largest effects observed only if a 
small proportion of the effect estimates represent real effects. Birnbaum 
(1959) investigated procedures related to half-normal plotting. His results 

on " the probabilities of the various possible sorts of errors..." are 

limited to the single largest order statistic. He surmised that if only a 
small number of effect estimates have nonzero means, the power and sensiti- 
vity properties will tend to hold approximately. 

The procedure of Wilk, Gnanadesikan, and Freeny (1963) if used with 2^ 

treatments is benefited if some subjective or prior knowledge is used to de- 

s. 

cide that n of the 2-1 mean squares do not contain real effects or 
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that p = 2*' - n -1 mean squares do contain real effects. As was shown by 
Wilk, Gnanadesikan, and Freeny (1963), their procedure is not robust against 
errors in guessing the value of n, and n must be guessed if prior knowl- 
edge is lacking. Chain pooling does not require prior knowledge of n. The 
procedure gives an estimate of n. 

Zahn (1975) presented results of a Monte Carlo study of Daniel ' s origi 
nal version of the half-normal plot and two modifications of it when as many 
as six real contrasts of size la to 8a are present in a where 

p - q =4, factorial experiment. Zahn considered the power, false positive 
behavior, and variance estimation of these versions of the half-normal plot 
when applied to the general problem in the case n =15, because there are 
15 contrasts of interest, ignoring the grand mean, in a 2^ factorial exper- 
iment. He then pointed out that these versions or obvious modifications of 
them can be used for any n. 

Zahn, Birnbaum, and Daniel limited their results to experiments where 
only a small proportion (at most 6 out of 15) of the effects are anticipated 
to be significant. On the other hand, situations can exist where the exper- 
imenter might use a two-level, fractional-factorial experiment designed such 
that most of the effects are significant. This situation is met by the pro- 
cedures of this report. 

If a two-level, full- or fractional-factorial experiment is performed 
and n^ observations are obtained from n^ orthogonal experimental condi- 
tions, the appropriate empirical equation for representing the results can 
have as many as n^ terms, each with a coefficient that has been fitted to 
the data. When this is done a question that should be asked is: Can the 
predictive accuracy be improved if some of the terms are deleted The fact 
that some of the terms might degrade the predictive accuracy of a fitted 
equation was recognized by Walls and Weeks (1969) who gave no procedure for 
identifying such terms. 

A method for the sequential deletion of terms that was intended to re- 
duce the prediction error was given by Kennedy and Bancroft (1971). Their 
method assumed that the experimenter has a prior establ ished order for sub- 
jecting the terms to a sequence of significance tests. Unfortunately, in 
many experimental situations there is no subject matter basis for establi- 
shing a prior order, and in such cases an order-statistics procedure is ap- 
propriate. An order-statistics approach for significance testing was used 
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in the related papers by Daniel (1959) and Birnbaum (1959). They were not 
then seeking to minimize prediction errors. 

For model-selection procedures used with small saturated experiments 
(experiments designed to have only as many experimental conditions as there 
are model parameters to be fitted), the analysis should begin with a minimum 
number of estimable terms being sacrificed to form a denominator for the 
test statistic. A procedure using m~terms sacrificed, where m can be as 
small as one, was investigated by Holms and Berrettoni (1969). The object 
was to delete terms in a manner where some control was maintained over the 
probabilities of Type 1 or Type 2 decision errors. 

The minimizing of prediction error was the object of an investigation 
of a chain-pooling strategy as described by Holms (1974). Whereas that in- 
vestigation had assumed only one cycle of analysis would be used, a sugges- 
tion by Holms and Berrettoni (1969) was that more than one cycle should be 
used. The purpose of a further chain-pooling investigation (Holms (1977)) 
was to optimize a combined procedure that might contain more than one analy- 
sis cycle where the procedure is to be optimized for minimum prediction 
error. An important application of chain pooling occurs in empirical opti- 
mum seeking. 

A widely accepted methodology for the design and analysis of experiments 
efficient for the empirical attainment of optimum conditions was introduced 
by Box and Wilson (1951) and refined by Box and Hunter (1957). That method 
is now known as response surface methodology. Designs that are optimal for 
fitting second-degree equations were studied by Lucas (1974 and 1976), who 
was concerned with the optimality of single-block designs, but multiblock 
designs are often appropriate in the applications of response surface 
methods. A catalog of multiblock designs, limited to those particularly 
applicable to response surface methods, was given by Holms (1967). Response 
surface methodology assumes that hypercube and star blocks contain center 
points. Criteria for the numbers of such points to use, together with tables 
of recommended numbers, were given by Box and Hunter (1957). Criteria lead- 
ing to much smaller numbers of center points for single-block experiments 
were given by Lucas (1976). The purpose of Holms (1979) was to characterize 
the experiment designer's options for numbers of center points in a range 
from very small to moderately large for multiblock sequential designs. The 
multiblock sequential designs were those for which treatment tables had been 
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given by Holms (1967). The numbers of center points used in each of the 
hypercube blocks ranged from zero to six. 

As stated by Birnbaum (1959), the optimal decision procedure when p £ 1 
uses a test developed by Cochran (1941). The Z| (originally in Yates' 
order) are ordered (omitting the mean square for the grand mean) according 
to nondecreasing magnitude and renamed 

Z(i) < Z(2) < ... < < ... < Z(^) 

Cochran's statistic is 

r ^(n) 

n + ... + 

and the null hypothesis is rejected with test size a if C^ exceeds the 
upper lOOa-percent point of Cochran's distribution. 

As pointed out by Zahn (1975), the operating characteristics of the 
decision procedures that have been intended for p £ 1 deteriorate rapidly 
with p > 1. But in addition to dealing with the situation of p > 1, the 
model-selection procedure must also deal with the possibility that not all 
the nonzero effects are of equal magnitude. As shown in table 8 of Zahn 
(1975), the detection rate for two means of size 4a, 6a, or 8a is usually 
reduced in the presence of two other means of size 2a, 4a, or 6a, respec- 
tively, from the value it would have been in the presence of two other means 
of the same size, namely, 4a, 6a, or 8a, respectively. Because inequality 
of the means is clearly a disadvantage, the question arises as to what is 
the least favorable distribution of these means. As developed by Holms and 
Berrettoni (1969), a normal distribution of these means would be highly 
unfavorable. 

Accordingly, the procedures of Holms (1980) were developed empirically 
to be optimal when the means do have an approximate normal distribution. 
Furthermore, the prior assumption of a normal distribution for these means 
in the general experimental situation does not seem to be unreasonable. 

Thus, the Monte Carlo founded procedures can be regarded as being empirically 
optimized against both what is highly likely to occur and what would be 
highly unfavorable if it did occur. Therefore, these procedures should be 


25 



regarded as being approximately both Bayes and minimax optimal. The proce- 
dures were developed to minimize the prediction errors in models fitted to 
the results of experiments having 16 hypercube points and any of zero to six 
center points. Results were exhibited as tables giving some of the operating 
characteristics of admissible strategies for each of the center point op- 
tions. A security regret strategy was identified within each set of admis- 
sible strategies, and it was shown to be useful for a wide range of coeffi- 
cients of variation. 


Admissible Strategies 

In the simulation studies of Holms (1980), the prediction errors Op^- 
of the deleted model equation were observed at each point of the hypercube, 
and the mean of the squared values ip^ was computed over the 1000 simulated 
experiments. The maximum of such error squares over the treatments was re- 
corded as 


_2 

®max 


2 

= max (Bp.) 


i = 1, 


_2 

The square root of e,„,„ was adjusted for the scale factor 

ITlaX 

simulations and for the number of treatments n^ where 


of the 


"t = "c 


using the equation 


r 1/2 - 

C = n. e 
ae,mx t max 


/e 


The quantity was then defined to be the adjusted coefficient of 

error. Its values were reported as functions of the scale parameter 6 
and as functions of the deletion strategy (mp, rp, «p, ay, r^). 

The ratio (in percent) of the single observation standard deviation for 
the simulations to the population value of the function for the largest 
value of the dependent variable was defined as the coefficient of variation. 
Scale parameter values of e of 0.125 and 2.0 resulted in coefficient of 
variation values of 64.3 and 4.0, respectively. In terms of laboratory 
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applications of response surface methods, these coefficients of variation 
were regarded as being large and small, respectively. 

Many strategies were investigated to determine a strategy that minimized 
for each of the values of n^, = 0, 1, 2, 3, 4, 5, 6 and for each 
coefficient of variation. Many strategies giving intermediate results were 
also investigated. A strategy was classed either as admissible or dominated 
according to its values of „„ at both v = 64.3 and v = 4.0. A 
strategy was said to be dominated if for v = 64.3 there were another stra- 
tegy with the same or lesser „„ for v = 64.3 and a lesser „„ 

GCjulX QCjfilX 

at V = 4.0. A strategy was also said to be dominated if there were a stra- 
tegy with the same or lesser „„ at v = 4.0 and a lesser „ 

Q. 0 jinx oicjinx 

at V = 64.3. Any strategy that was not dominated was defined as being 
admissible . The strategies that were found to be admissible, together with 
some of their operating characteristics, were tabulated by Holms (1980). 

Security Regret Strategy 

The order of the tabulations for each value of Oq in Holms (1980) was 

in the order of decreasing values of C at v = 64.3, which is in the 

Qw j I \ \/\ 

order of increasing values of C at v = 4.0. 

G0 } I FIX 

The regret function of a statistical decision procedure as a function of 
a parameter v is here defined as the excess loss occurring with the proce- 
dure at a particular value of v, as compared with the loss that would have 
occurred had the best statistical decision procedure been used for that par- 
ticular value of V. For the purposes of the present investigation, a regret 
function R(v) is defined for v = 64.3 as being the (64.3) for any 

^ ~ Q 0 j mx 

strategy divided by the value of for the best strategy for that 

o 0 j mx 

value of v; R(v) is defined for v = 4.0 as being the C,. mv(4*0) 

strategy divided by the value of for the best strategy for that 

u 0 ^ 1 nx 

value of V. 

Thus, for the successive values of Uq, the regret functions R(Oq,v) 

are 


R(nQ, 64.3) = 64.3)/min[C^^ 64.3)] 


"ae,mx^ 0 


ae,mx^ O' 


and 
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R("0’ '*■'» ■ '^ae,ir.x<'’0’ 

From table 4 of Holms (1980), the values of min[C^_ ;„„(rin, 64.3)] and 

^ ITIa Vy 

mv(>^n> 4.0)] were as follows: 

a G > in A U 


no 

min[Cae,mx(no. 64.3)] 

min[Cae,mx(no. 4.0)] 

0 

29.46 

2.065 

1 

30.39 

2.118 

2 

31.11 

2.187 

3 

31.95 

2.244 

4 

32.63 

2.300 

5 

33.51 

2.352 

6 

34.25 

2.401 


The single strategy that has the smallest regret function over both 
V = 64.3 and v = 4.0 is defined as the security regret strategy . The 
security regret strategy is thus the sequential deletion procedure which, 
for a given nQ, produces the least increase in prediction error (for 
p = 15 and an unfavorable distribution of parameters) over that prediction 
error which could have been achieved if the best strategy had been chosen 
for the given (unknown) value of error variance a . 

In examining the R(v) values of table 4 of Holms (1980), for a given 
value of Oq, the parameters that give the security regret strategies are 
those that give the joint minimums on R(64.3) and R(4.0); these joint 
minimums were identified by asterisks. Thus, for the, given values of n^, 
the security regret strategies and the associated values of C 3*"® 

as follows: 


^0 

m 

P 


“f 

“U 

r 

n 

C (64.3) 

ae,mx' ' 

C (4.0) 

ae,mx' ' 

0 

1 

— 

1.0 

0.50 

0.25 

32.95 

2.240 

1 

0 

3.0 

.50 

.10 

.80 

31.78 

2.180 

2 

0 

3.0 

.25 

.50 

.85 

33.14 

2.252 

3 

0 

.0 

.75 

.50 

.80 

33.29 

2.301 

4 

0 

.5 

.50 

.50 

.80 

33.89 

2.309 

5 

0 

1.0 

.25 

.10 

.80 

33.97 

2.394 

6 

0 

.5 

.50 

.05 

.80 

34.72 

2.408 


The question can be asked as to what choice of Oq will result in the 
most efficient experiment. If the object of a choice of Oq is to use the 
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most efficient choice together with a security regret strategy for deleting 
terms, then the preceding table shows that the most efficient choice (the 
choice that minimizes each of C ®nd C the choice 

of Oq = 1). This choice applies to the condition of n^, = 16. 

Selection of a Strategy 

Results for the strategy that minimizes mv(4.0), for the security 

d 6 ^ m X 

regret strategy, and for the strategy that minimizes C „^(64.3) are pre- 
sented in that order for each value of Oq in table 1(c). 

In summary, if the experimenter wishes to minimize the maximum predic- 
tion error over the 16 hypercube points of an experiment with Og center 
points when the variance error is relatively large (coefficient of variation 
in the range of 65 percent), the strategy for a given ng should be the 
last-listed strategy (for the given ng) of table 1(c). If the experimenter 
wishes to minimize the maximum prediction error over the points of the exper- 
iment when the variance error is relatively small (coefficient of variation 
in the range of 4 percent), the strategy for a given ng should be the 
first-listed strategy (for the given ng) of table 1(c). 

If the experimenter has no basis for a choice of one of the two preced- 
ing extreme choices, the choice should be a security regret strategy as indi- 
cated by the b results in table 1(c) where (for all of the ng values) the 
largest value of the regret function was R(64.3) = 1.1185 as listed in table 
1(c). This value of the regret function shows that for the worst value of 
ng(ng = 0) the relative prediction standard error was increased by at most 
about 12 percent over what it would have been if the worst value of v had 
occurred and the best strategy against it had been used. Thus, the security 
regret strategies (for each of the values of Og) must be concluded to be 
widely useful strategies. 

Estimates of Standard Deviation 

Two procedures are used to estimate the single observation standard 
d(‘viaLion. The first provides an estimate Sj|^ obtained prior to any model 
deletion, for the six cases listed in table 1(a) the estimates are computed 
according to 
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Case a 
Case b 
Case c 
Case d 
Case e 
Case f 



As illustrated in appendix E, the number of degrees of freedom and the value 
of Sj are printed following the printing of Y(I). 

Subsequent to the termination of the sequential deletion procedure, the 
accumulated number of degrees of freedom n^^ and the accumulated sum of 
squares SS are used to estimate a standard deviation as follows: 


S2 = (SS/n^^)^/^ 


The values of n^^ and S2 are printed following the listing of the coeffi- 
cients B(I). 
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APPENDIX B 


DERIVATION OF EQUATIONS FOR. SUMS OF SQUARES 

Case a (Hq = 0, nip = 0) . - This case provides no information for a sum 
of squares for a test statistic. 

Case b (no = 1, mp = 0). - Let Yok be the value observed at the 
center point (origin) observation. Then, by the definition of 

V(Yq^) = (Bl) 

Also from the definition of where Y^. is the i^*^ hypercube observation, 

V(Y.) = (B2) 

p 

The object is to estimate a from the information in the y^. , i = 1, 

...., n^,, and a single center point observation yQp Because the model 
coefficient estimates in the two-level fractional-factorial experiment are 
orthogonal, the least-squares estimates of the regression coefficients from 
the combined data are all the same as the Yates estimates, except for the 
coefficient of the zero degree term b^. Its least-squares estimate is from 
equation (5): 

“■l ■ ^^01 ^ E 4/'^ * "c> 

For any of the treatment points, let be the difference between the 
observed value and the predicted value of Y where Aq is the center point 
difference and i =0, 1, ..., n^. The Yates estimate of is 



The predicted values under the least-squares estimation are therefore all 

ic 

augmented by b^ - b^ over their Yates method predictions. 
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The differences between the Yates method predictions and the observa- 
tions are all zero at the hypercube points; therefore, over the n^ +1 
treatment points 

^0 = ^01 " 

i=l5*.«5n^ 

2 

The estimate of a from the residual of the least-squares regression is 



s = 


E 

i=0 


+ 1 - 
c c 


(yoi - bj) 


•A- 9 . ? 

+ n (b, - b,)-^ 


where 


and 


^01 " " -^01 


He 

^01 E ^1 ’'c^i - E 

i=l i=l 


1 + n. 


nc n^ n^ 

E ^01 " E E - "c^i 

* _ i=l i=l i=l 

1 ""1 “ n 1 + n “ n (1 + n ) 

c c c c 


b, - b 


Thus, 


s^=(yol^b;)2*„c(bi-l,;)2=T^(y„j-bi)2 


For n = 16, 
c ’ 


s^ = 0.941176(yQ^ - b^)^ 


For this case, the number of degrees of freedom is 
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Vb “ ^ 


(B3) 


and the sum of squares is 


" ^I'^dfb “ 0.941176(yQ^ 




(B4) 


In this case, the error sum of squares was obtained from a residual involving 

ic ic 

bj^. This usage of b^ has the disadvantage that it will introduce a bias or 
lack-of-fit component into the sum of squares if the fitted model is biassed 
at the center point. Because of this bias risk, a pure error sum of squares 
will be computed if Oq > 1. 

Case c (n^ > 1, m = 0). - Let the center point observations be yoi<; 

b! b! 

k = 1, ..., ng. Their sample mean is 

1 

^0 " n. £ •^Ok 
^ k=l 

and the sum of squares SSj; for case c is now: 


k=l k=l ^ \k=l 


^Okj 


(B5) 


where the number of degrees of freedom nqf^ is 


"dfc • "o - 1 


(B6) 


Case d (Oq = 0, mp > 0). - Let 


5'd- E ^(j) 

j=l 


(B7) 


The number of degrees of freedom nqfq is 


"dfd = "'p 


(B8) 


and d: 


Case e (n^ = 1 , fflp > 0). - This is the additive situation of cases b 
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ss = ss, + ss , 

e b d 


•^dfe " "dfb '^dfd ' ^ "’p 


Case f (iIq > 1, nip > 0). - This case is additive with respect to 


cases c and d: 


SS. = SS + SS , 
fed 


"^dff “ "dfe Vd " % ^ "’p 
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APPENDIX C 


OPERATING SECTIONS OF PROGRAM POOLIO 

The input (section 1) and output (section 5) of the program were des- 
cribed in the main text of this report. This appendix describes the operat- 
ing sections of the program, namely, sections 2, 3, and 4, with sections 3 
and 4 illustrated by the flow chart of figure 3. The program was tested on 
a Univac 1100 system. 

Section 2 - Yates' method . - The algorithm for Yates' method is des- 
cribed as follows: The observations y. • may be visualized as a column 

I ' » J 

(j = 1) with row index i = 1, ..., 2 . The column is then operated on ac- 
cording to Yates method to produce a succession of columns j = 2, ..., n- . 
The successive columns for any row are computed as follows: 

fi = 1, 3, 5, ..., 2^ - 1 

^k, j " ^i+1, j-1 ^i ,j-l T 

t^k = (i + l)/2 

p = 1, 3, 5, ..., 2*" - 1 

^k,j " ^i+l,j-l “ •^i,j-l 1 ^ 

[k = (2 + i + l)/2 

New columns are computed according to the two preceding equations for 
j = 2, ..., i. (to create it columns). 

Section 3 - Construction and ordering of mean squares . - The mean 
squares are formed from the parameter estimates (for those terms beyond 
3^) and a pointer function is created within the loop DO 309, I = 1, NO. 

The array of pointers to the B(I) array is created by the statement 
IND(I) = I. This array will serve to identify the coefficients in the B(I) 
array after the process of ordering mean squares according to rank. The 
ordering is done in the sequence of statements ending with 313. 

Operations thus far created a column of mean squares Z(J) with mean 
squares indexed on J in the order of increasing rank, together with a col- 
umn of integers IND(J) indexed on J. Thus, any address J will lead to a 
mean square Z(J) and also to the integer IND(J). This integer is the 
index I that the associated regression coefficient has in the original 
Yates order. 
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The computation of the sums of squares begins following statement 313 
and ends with statement 355. The operations are outlined by figure 3(a). 

The computation of the sums of squares depends on the values of nQ and 
mp according to cases b, c, d, e, and f of table 1(a). Three combinations 
of these cases are identified in the statement immediately preceding state- 
ment 320. If ng = 0, the situation is that of case a or d and control is 
transferred to statement 330, and, subsequently, the SS^ of equation 
(B7) is evaluated at statement 365. 

If ng = 1, the situation can be that of case b (ng = 1, mp = 0) or case 
e (ng = 1, mp > 0) and the SSj^ of equation (B4) is computed at the state- 
ment following 325. 

If ng > 1, the situation can be that of case c or case f. The quantity 
SS^ of equation (B5) is computed at the statement for TEM that follows 
statement 322. 

In terms of mathematical symbols previously defined, the strategy para- 
meters are functions of numbers that are read at statement 362 as follows: 


Argument 
FORTRAN symbol 

Function 

Mathematical symbol 

MP 

"p 

RF 

"'f 

KPF 

“F 

KPU 

“u 

RETA 

r 


n 


More than one model-deletion strategy can be used during any computer 
run. Upon completion of the use of a particular strategy, control is trans- 
ferred back to statement 362 for the reading of an additional strategy data 
card. The operation of the program ends when such cards are exhausted. 

If mp and ng are each zero, there can be no sequential deletion; 
therefore, control is transferred to statement 432, and all terms are retained. 
Setting both and ag equal to 1.00 (by setting KPF = KPU = 11) is 
used as a code to signify that no conditional pooling is to be done but that 
arbitrary deletion is to be accomplished according to values assigned to m 

P 

and r^. This is done by transferring control to statement 421. 
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Section 4 - Deletion of terms. - The flow chart for the tests of signi- 
ficance is shown by figure 3(b). The procedure ends at statement 419 (appen- 
dix D). The significance tests will have been avoided by earlier statements 


in section 3 if either n^ + m =0 or both ar = 1.0 

Up F 


and tty = 1.0. 


Thus, sequential deletion requires both fig ^ '"p ^ ^ least one of 


1.0, control is transferred to the U- test 

J 


otp or ^ 1.0. If ^F 

which begins at statement 418. If op < 1.0, control is determined (fig. 
3(b)) by the questions: Is j > rpny? Is j > 1? If both are yes, control 
is transferred to statement 418 which initiates the Uj-testing. 

Irrespective of whether significance testing is against the F-distribu- 
tion or the Uj-distribution, insignificance pools into the denominator 

of the test statistic and then transfers control to statement 419 which in- 
creases j by one unit. Significance at any j transfers control to 
statement 420. 

The third statement in section 4 (namely, IF (KPF. GT. 10) GO TO 418) 
transfers control to the U .-test merely provided op = 1.0, even if j < 2. 
But the rationale of the U —distribution leaves U- undefined for j < 2. 

vJ 0 

The possibility of a transfer of control to the U— distribution with j = 1 

J 

was provided for by setting the critical values of U - equal to 2.0 for 

J 

j = 1 and all values of «y < 1.0. Thus, if j = 1, then obviously 
mp = 0 and the test statistic is (from table 1(a)) 


2.0z 


^1 ~ SS, 


( 1 ) 


2.0 


b ^(1) 




for case b (ng = 1, mp = 0) or 


1 


for case c (ng > 1, irip = 0). Thus, for case b, u^ £ 2.0 for SSg ^ 0.0 
and would not test as significant. For case c (ng > 1, mp = 0), 

uj > 2.0 only if 


SS 


'^ 0 ^( 1 ) 
c + ^(1) 


> 2.0 


hence only if 
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2.0SS^ 

^(1) > (Pq - 2.0)' 

Let a2 be estimated by SS(-l{nQ - 1). Then > 2.0 only if 

2.0(nQ - 1) SS^ ^2 

^(1) ^ (nQ - 2.0) (nQ - 1) > 

The following table shows C(ng) as a function of Oq for the values of ng 
appropriate to case c: 


^0 

C(no) 

2 

QO 

3 

4.0 

4 

3.0 

5 

2.67 

6 

2.5 


Thus, (the smallest of the ordered mean squares) would have to be much 
larger than before would be declared significant. 

The flow chart for the model deletion and for the estimate p is shown 
by figure 3(c). Transfer of control to statement 420, 421, or completion of 
the loop at 419 leads to the estimate, respectively. 


or 


n = maximum integer £ r^(j - 1) 


or 


With n 


n = maximum integer £ r^^mp 
n = maximum integer £ '"n('^c “ 

so estimated, the n smallest absolute value coefficients (beyond 


b;[) are set equal to zero with the statements ending at 425. 
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APPENDIX D 


LISTING OF COMPUTER PROGRAM POOLIO 


lA.- DECLARATIONS AND TABLES 


C 


C 


C 


DIMENSION REMARK(l«n, ALPHAUl), FTB(20,10», TBtl6,10), DATAID<24» 
A, YIN(2‘4,6)» IND(16), Z(16)t Y (24), B(16), B0UT(16) 

data ( ALPH A(I ) ,1=1,1 1 )/0. 001 ,0. 002, 0.005,0.01 , 0.025, 0.05,0.10, 

10.25,0.50,0.75,1.0/ 


DATA ( (FTB (1 ,J ) ,J = 1, 10), 1 = 1,20 )/4 05 280. 0,101321 .3, 16211. 0,4 052.2,64 
A 7. 8, 161.4, 39.86,5.828,1 .000,0 .17 16,998 .5,49 8.5, 19 8.5,98.50,38.51, 1 
B 8. 51, 8.526,2.571 ,0. 6 667,0. 1333 ,167 .0,104.3,55.55, 34.12, 17.44, 10.1 3 
C« 5. 538, 2.024,0.5851 ,0.1220,74 .14 ,5 1 .45 ,31.33,21.20,12.22,7.709,4. 5 

045. 1.807. 0. 5486.0.1165.47.18.34 .73 .22.78 .16.26. 10.01. 6.608.4.060. 1 

E . 692,0.5281,0.1134,35. 51, 27. 12, 18. 64, 13. 74, 8. 813, 5. 98 7, 3. 7 76, 1.621 

F , 0.5149,0. 1113,29.24 ,22.90,16 .2 4 ,12.25 ,8.07 3, 5.5 91,3.58 9,1 .573,0. 5 

6057.0. 1099.25.42.20.26.14.69.11 . 26 . 7 .5 7 1 . 5 . 31 8 . 3 . 458 . 1.538.0.4990, 
HO. 1088, 22.86, 18.46,1 3. 61 , 1 0 . 56 , 7 .2 09 , 5 • 1 1 7 , 3. 360 , 1 . 5 1 2, 0.4938 ,0.10 
180, 21.04, 17.17, 12.83,10.04 ,6. 93 7 ,4 .965 ,3.285, 1 .4 9 2, 0.4897, 0. 107 3, 1 
J 9. 69, 16.20, 12.23,9.646,6.724,4.844 ,3.225,1.475,0.4864,0.1068, 18.64 
K, 15.44, 11 .75,9.330 ,6. 554 ,4.747,3.176,1 .461 ,0.4837,0. 1063,17.82,14. 
L84, 11.37, 9.074,6. 414,4. 667 , 3 . 1 36 , 1 . 45 0 , 0 . 48 14 , 0 . 1059, 1 7 . 14 , 1 4 . 34, 1 
Ml. 06, 8. 86 2, 6. 298, 4, 600, 3. 102, 1.4 403 ,0 . 479 4 , 0. 1 056 , 16 . 59 , 13 .94 , 1 0. 8 
NO, 8. 68 3, 6. 200, 4. 54 3, 3. 0 73, 1.4 32 ,0.4778 ,0. 1 053 , 16 . 12, 1 3. 59, 10.58,8.- 

0531. 6. 115.4. 494. 3. 048. 1.425. 0. 47 63. 0.1 05 1.15. 72. 13. 29. 10. 38. 8. 4 00, 
P6. 04 2, 4. 45 13, 3. 026, 1.41 9,0. 4750, 0.1 049, 15. 38, 13.0 3, 1 0 .2 2, 8 .28 5 , 5. 9 
Q 78, 4.414, 3.007, 1.41 30 , 0 . 4 73 8 , 0 . 1 04 7 , 15 .08 , 1 2 . 8 1, 1 0 . 0 7, 8 . 18 5 , 5 .9 22 , 
R 4. 38 1,2. 9 9 0, 1 .408,0. 4 72 8,0.1045 , 14 .82, 12.62 ,9.944,8.096,5.872 ,4.35 

51. 2.975. 1.404.0. 4719.0.1044, 

DATA ( ( TB (I ,J ) ,J=1,10 ) ,1 = 1 ,16 >/2 .0, 2.0,2. 0,2. 0,2. 0,2.0, 2. 0,2.0, 2.0 

1 .2.0, 2.0000,1 .99999,1.9 9997,1 .99986 , 1 . 999 1 7 , 1 . 99 6 87, 1 .9 877, 1.923, 1 
2 .70 6, 1.38 2,2.9976,2. 9960 , 2 . 9904 , 2 . 9 809 , 2 . 9 5 1 , 2 .904 , 2 . 80 6, 2 . 52 7, 2. 0 
3 86, 1.68 8, 3.976,3. 962,3. 925 , 3 . 87 0 ,3 . 76 0 , 3 . 625, 3 . 4 1 2, 2 . 94 9, 2 . 39 5 , 1. 9 
461,4.887, 4. 845, 4. 758, 4. 65, 4. 44, 4 .2 1 , 3 . 89 , 3 . 28 7 , 2 . 658 , 2. 184,5.74,5. 
563, 5. 46, 5. 31, 4. 99, 4. 68, 4. 28, 3. 57 , 2. 89 3, 2. 371, 6. 51, 6. 3 3, 6. 11, 5. 87, 5 
6.46,5.09, 4.61,3.83,3.11 ,2.54 ,7.20,6 .96 ,6.65,6.35, 5.88,5.44,4.91,4. 
7 06, 3. 29, 2. 69, 7. 81, 7. 52, 7. 10, 6 . 78 , 6. 26, 5. 75, 5. 17, 4 . 27, 3. 45, 2. 82, 8. 3 
84, 8. 01, 7. 53, 7. 17, 6. 59, 6. 03, 5. 41 , 4. 45, 3. 60, 2. 95, 8. 82, 8. 4 4, 7. 95 , 7. 53 
9, 6. 89, 6. 28, 5. 61, 4. 62, 3. 74, 3. 07, 9, 26 , 8. 84, 8. 33, 7. 8 7, 7. 13, 6. 50, 5. 81 , 
A4. 77, 3. 87, 3. 17, 9. 67, 9. 21, 8. 68 , 8. 16, 7. 37, 6. 71, 5. 99, 4. 9 2, 3.99,3.27, 1 
BO. 05, 9. 55, 8. 95, 8. 42, 7. 59, 6. 91 ,6. 15, 5. 05, 4. 10, 3. 37, 10. 40, 9. 86, 9. 20, 
C8, 66, 7. 79, 7. 07, 6. 30, 5. 17, 4. 20, 3. 46, 10. 72, 10. 14,9. 43, 8. 8 3, 7. 96, 7. 23 
0 ,6.44,5.29,4.30,3.55/ 
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IB.- INPUTS AND CONSTANTS 


C 
C 
C 

REA0(5,8O0) tREMARK CI)»I=1 *1« ) 

WRITEC6»801) (REMARK ( I » ,1 = 1 .1*1 ) 

READ (5, 8021 NO, KODE 
KK = 4 
NC = 16 

WRITE (6,8031 NC, NO, KODE 
NCMl = 15 
NCM2 = 14 
NCPl = 17 
NT = NC ♦ NO 
FNC = NC 
FNO = NO 
WRITE (6,8051 
DO 103 1=1, NT 
NY IN = 1 

READ (5,804) OATAID(I), NYIN, (YIN(I,K), K=1,NYIN) 
WRITE (6,807) OATAID(I), (YIN(I,K), K=1,NYIN) 

YII) = 0.0 
FNYIN = NYIN 
DO 102 K=1,NYIN 
IF (KODE .LT. 1) GO TO 102 
YIN(I,K) = AL0610(YIN(I,K ) ) 

102 Y(I) = Y(I)*YIN(I,K) 

Yd) = Y(I)/FNYIN 

103 CONTINUE 
WRITE (6,813) 

DO 105 1=1, NT 

WRITE (6,815) DATAID(I), Y(I) 

105 CONTINUE 

2.- YATES METHOD 

II = 2**KK 
IIDB2 = 11/2 
KKMl = KK-1 
00 208 K=1,KKM1 
00 206 1=1,11,2 
IP1D2 = (I+D/2 
B(IP102) = Y(I^1)*Y(I) 

LL = IP1D2+IIDB2 

206 B(LL) = Y(I*1)-Y(I) 

DO 207 1=1,11 

207 Y( I) = B(I ) 

208 CONTINUE 
DO 209 1=1,11,2 
IP1D2 = (l*l)/2 
B(IP1D2) = Y(I*l)*Y(I) 

LL = 1P1D2*IIDB2 
B(LL) = Y (!♦! )-Y(I ) 

209 CONTINUE 
C 
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C 3.- CONSTRUCTION AND ORDERING OF MEAN SQUARES 

C 

DO 309 1=1, NC 
IND(II= I 

ZII> = B<I*lJ#Ba*l )/FNC 
BID = B(I >/FNC 
309 CONTINUE 
C 

DO 313 J=1,NCM2 
TEST = ZINCMl) 

IN = NCMl 

DO 312 NA=J,NCH2 

IF! TEST-ZINAl) 312,312,311 

311 TEST = ZINA) 

IN= NA 

312 CONTINUE 
ITEM= INDIIN) 

TEM= ZIIN) 

INOiIN)= INDIJ) 

ZIIN)= ZIJ) 

INDIJ) = ITEM 
ZIJ) = TEM 

313 CONTINUE 
C 

TY = 0.0 

IF INO -1) 330, 325, 320 
320 SSQYO = 0.0 

DO 322 I=NCP1,NT 
TY = TY ♦ Y II) 

SSQYO = SSQYO ♦ Y II)**2 
322 CONTINUE 

TEM1= SSQYO - I ITY**2 l/FNO ) 

NDFir NO -1 

BID = IFNC*BI1) ♦ TY)/(FNC ♦ FNQ) 

FNDF = NDFl 

SI = ISQRTITEMl) )/FNDF 
60 TO 355 
325 TY = Y INCPl) 

TEM1= 0.941176*l ITY - B ll))*«2) 

NDF1= 1 

BID IFNC*BI1) ♦ TY)/IFNC ♦ FNO) 

SI = SORT I TEM 1) 

60 TO 355 
330 TEM1= 0.0 
NDF1= 0 
SI = 0.0 
C 

355 WRITE 16,817) NDFl, SI 
C 

C STRATEGY 

C 

362 READ 15, 808, END - 899) HP ,KPF ,KPU,RF,RETA 
MPPl = MP*1 
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NDF = NDFl 

TEM = TEMl 

RFNO = RFOFNO 

IF (HP .LT. 1) GO TO 370 

DO 365 J = 1,MP 

TEM = TEM ♦ Z(J» 

365 CONTINUE 

NDF = NDF ♦ MP 
FNDF = NDF 

S = < SQRT(TEM ) l/FNDF 

370 IF (NO .LT. 1 .AND. MP .LT. I) GO TO 432 

IF (KPF .6T. 10 .AND. KPU .GT. 10) GO TO 421 

4.- DELETION OF TERMS 

DO 419 J=MPPltNCMl 
JN = J 

IF (KPF .GT. 10) GO TO 418 
FJ = JN 

IF (FJ .ST. RFNO .AND. JN .GT . 1) GO TO 418 
F - TEST 
FNDF - NDF 

TEST = FNDF ♦ 2(J) / TEM 

IF (TEST .GT. FTB (NDF ,KPF) ) 60 TO 420 

TEM = TEM •* Z(J) 

NDF r nDF •» 1 

GO TO 419 

UJ - TEST 

418 IF (KPU .GT. 10) GO TO 420 
NDFPl r NDF ♦ 1 

FNDFPl r NOFPI 

TEST = FNOFPl+2 (J )/ ( TEM+ZI J) ) 

IF (TEST .GT. TB(JN,KPUn GO TO 420 
TEM r TEM ♦ 2(J) 

NDF = NDF ♦ 1 

419 CONTINUE 
JETA = NCMl 
GO TO 422 

4 20 JETA = JN-1 
GO TO 422 

421 JETA = MP 

422 ETA r JETA 

JETA r IFI XIRETA^ETA ) 

FNDF r NOF 

S r ( SORT (TEM n/FNDF 
GO TO 434 
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432 JETA = 0 
NDF = Q 
S r 0.0 

434 JA = JETA ♦ X 

IRHO = NCMl - JETA 
DO 424 I=XtNC 
BOUTm = B(I ) 

424 CONTINUE 

IF (JETA ,LT. XI GO TO 505 
DO 425 J-X,JETA 
INDX = IND(J» ♦ X 
BOUT(INOX) = 0.0 

425 CONTINUE 

5.- OUTPUT 

505 WRITE (6,809) (HP, RF, ALPHA(KPF)» ALPHA(KPU), RETA) 

WRITE (6, SIX) IRHO, JETA, Z(JA) 

WRITE (6,82X) 

00 5X0 I=X,NC 
INDXI = I 

WRITE (6,8X9) INDXI, BOUT(I) 

5X0 CONTINUE 

WRITE (6,8X7) NDF, S 
GO TO 362 
899 STOP 

800 FORMAT (X3A6,A2) 

SOX FORMAT ( XH X , // XOX , X 3 A6 , A2/ / ) 

802 FORMAT (2IX) 

803 FORMAT (XH0,3X,4HNC =I5,5X,4HN0 =15 ,5X ,6HK00E =12) 

804 FORMAT ( A6 , 16 ,6F6. 0 ) 

805 FORMA T (XH0,6H0ATAID,5X,4HY(I) ,9X ,4 HY ( 2 ) , 9 X4H Y ( 3 ) , 9X , 4H Y ( 4 ) , 9X , 4H Y 
A(5),9X,4HY(6) ) 


8 0 7 F ORmA T ( X X , A6 , 6E X 3. 5 ) 

808 FORMAT (3I5,2F5.3) 

809 FORMAT ( XH X// , XX, 4HHP =I5,5X,4HR.F =F6 . 3, 5X , 8HALPHAF =F6 . 3, 5X , 8H AL P 
AHAU =F6.3, 5X,6HRETA =F6.3) 


8XX 

FORMAT 

8X3 

FORMAT 

8X5 

FORMAT 

8X7 

FORMAT 

8X9 

FORMAT 

82X 

FORMA T 


END 
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APPENDIX E 


ILLUSTRATIVE OUTPUT OF COMPUTER PROGRAM POOLIO 


SECOND factorial EXPERIMENT, FULL REPLICATE OF 2**4 04-02-82 


nC r IS NO = 4 K ODE = 1 


DAT AID 

Yd) 

1 

.12670+03 

2 

.19600+03 

3 

.16340+03 

4 

.19400+03 

5 

.889 00 + 02 

6 

.17580+03 

7 

.15490+03 

8 

.144 10+03 

9 

.136 10+03 

10 

.65700+02 

11 

.12980+03 

12 

,806 00+02 

1 3 

.17580+03 

14 

.16720+03 

15 

.14180+03 

16 

.14500+03 

C-1 

.19520+03 

C-2 * 

. 1 14 30+03 

C-3 

.149 50 + 03 

C-4 

.15870+03 

DATAID 

Yd) 

1 

.21748+01 

2 

.22787+01 

3 

.21984+01 

4 

.23423+01 

5 

.19873+01 

b 

.22247 +01 

7 

.22253+01 

8 

.21846+01 

9 

.208 16+01 

10 

.17979+01 

1 1 

.207 17+01 

12 

.19247+01 

1 3 

.22310+01 

14 

.22218+01 


Y(2) Y(3) 

,17650^D3 
.18410+03 
. 15260^03 
,24940+ 03 
.10610+03 
,16010+03 
.18220+03 
.16240+03 
,10700+03 
.60000+02 
.10720+03 
.87700+02 
.16480+03 
,16610+03 
.12920+03 
.14010+03 
.18560+ 03 
.88000+02 
,11980+03 
.14320+03 


Y(4) Y(5) Y( 
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i .21315*01 

. .21539+01 

■1 .22795+01 

•2 .20013-^01 

-3 .21265*01 

.21783+01 

NDF :: 3 S = .66830-01 

- 0 RF - .500 ALPHAF r . 5 qo ALPHAU = .500 RETA r 

IRHO z 6 JETA = 9 Z(JETA+1) = .11194-01 

TAIO B(I) 

1 .21408*01 

2 .00000 

3 .00000 

4 .00000 

5 .30628-01 

6 .00000 

7 .00000 

8 -.26450-01 

9 -.62629-01 

a -.53884-01 

1 ,00000 

2 .00000 

3 .77157-01 

4 ,30950-01 

5 .00000 

6 .00000 

NDF =15 S = .23759-01 


.800 
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APPENDIX F 


SYMBOLS 


Mathematical 

FORTRAN 

Description 

symbol 

name 


^i 

B(I) 

estimate of 3^ 

*"ae,mx 


adjusted coefficient of error 

E(. . .) 


expectation of . • • 

e 


single observation random error 

2 

0 


maximum over hypercube of mean square 

max 


prediction error over simulations 

g 


number of independent variables 

g-h 


experiment contains 2^“*^ treatments 

h 


experiment contains (1/2)^ times 


number of treatments in full- 
factorial experiment 


i 

I 

subscript denoting order of computing 
mean squares according to Yates' algo 
rithm: i = 1 , 2 , . . . , n^, I = 1 , 

NC 

j 

J 

subscript denoting the j smallest 
mean square (exclusive of grand mean) 
j — T> 2, •••9 

k 

K 

subscript 


KPF 

index number for op 


KPU 

index number for ay 

"P 

MP 

number of mean squares pooled before 
testing begins 


NYIN 

number of observations averaged for 
each treatment (maximum of six) 

"o 

NO 

number of center points 

n 

c 

NC 

number of hypercube points 

"t 

NT 

total number of observations in one 
experiment 


RF 

distribution transfer parameter 
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< c: ;> a < a 


r 

n 


V(. . .) 


Y 

A 

Y 


“F 

“U 


n 


a 


RETA 

TB 


YIN(I,K) 

Z(I) 


JETA 


IRHO 


number of terms deleted is integer value 
of r times number insignificant 

n 3 

test statistic 
variance of . . . 

independent variable 
conceptual value of dependent variable 
estimate of response function from 
fitted model 

response variate: I =1, NT 

K = 1, NYIN 

mean squares in Yates' order 

nominal significance level of F test 

nominal significance level of U- test 

J 

coefficients of model equation that are 
estimated in Yates order 
number of mean squares having noncen- 
trality parameter of zero 
number of terms deleted 
coefficient of variation 
number of real effects (not including 
zero degree coefficient) 
number of terms retained (beyond x®) 
standard deviation of e 
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TABLE 1. 


CHAIN-POOLING DELETION 





(a) 

Expressions for degrees of 

freedom , 

sums of squares, and test statistics 




"dfO 



Eq. no. 

F-Test (see eq. (14)) 

i j 1 

U -Test (see eq. (16)) 

(1 < j < nj,- 1) 

0 

0 

I 


SSb = 

0.94U76(yoi - 

(B4) 

(1 + j - Dz(j) 

("dfo 

1 

0 

1 

i>j ■ SSb + z(D ■*■•••+ Z(j-D 

“j SSb ^(1) +•••■*■ Z( 

>1 

n 

n - 1 
0 

SSc = 

^ /"O \2 

^ yok - 7^ ( ^ yok) 

k=l 0 \k=l / 

(B5) 

(no - 1 + j - 

Wfo J)^(j) 


^l.nQ+j-2 SS^ + Z(j^) +. ..+ 2(j_i) 

j SS^ + +. . .+ z 

0 

>0 

“P 

SSd = 

? 

A 

J=1 

(B7) 


j"(j) 

SSj + 2(mp+D 

1 

>0 

1 + nip 

SSe = 

SSb + ss^ 

(B9) 

(1 + j - Dz(j) 

(1 + j)Z(j) 

SSg + Z(rop+i) +••.+ z(j_i) 

"J " SS^ + ^(m[H-D 

>1 

>0 

"0 " 1 + ”*p 

SSf = 

SSc + SS^ 

(BID 

(no - 1 + J - Dz(j) 

(no - 1 + j)z(j) 

f 

l,nQ+j-2 SSf + Z(„p+i) +...+ 

“j SSf + Z(„p4_j^)+...+ 



TABLE 1. - Continued. 

(b) Values of j at which transfer from F- to Uj-distribution occurs 



"o 


0 

1 

2 

3 

4 

5 

6 


j 

0.0 

2 

2 

2 

2 

2 

2 

2 

.4 

2 

2 

2 

2 

2 

3 

3 

.5 

2 

2 

2 

2 

3 

3 

4 

.7 

2 

2 

2 

3 

3 

4. 

5 

.9 

2 

2 

2 

3 

4 

5 

6 

1.0 

2 

2 

3 

4 

5 

6 

7 

2.0 

2 

3 

5 

7 

9 

11 

13 

4.0 

2 

5 

9 

13 

n.t. 9 

n.t. 

n.t. 

8.0 

2 

9 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 

16.0 

2 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 

n.t. 


3No transfer occurs, n.t. 
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= 64.3 


1.1215 

1.1185 

1.0000 

1.0875 

1.0457 

1.0000 

1.1058 

1.0653 

1.0000 

1.0770 

1.0419 

1.0000 

1.0536 

1.0386 

1.0000 

1.0719 

1.0137 

1.0000 

1.0736 

1.0137 

1.0000 









TABLE 1. - Continued 


(d) Results for F-only strategies 


■^0 

“F 

P 


C (v) 

ae,mx' 

R(v) 



V = 64.3 

V = 4.0 

V = 64.3 

V = 4.0 

V = 64.3 

V = 4.0 

1 

0.05 

.10 

3.760 

6.576 

7.627 

11.58 

38.09 

32.99 

21.60 

6.403 

1.0000 

3.0189 


.25 

9.858 

13.68 

33.20 

2.263 

1.0064 

1.0669 


.50 

11.87 

14.40 

33.63 

2.166 

1.0194 

1.0212 


.75 

13.16 

14.72 

33.89 

2.122 

1.0273 

1.0005 


1.00 

15.00 

15.00 

33.94 

2.121 

1.0288 

1.0000 

2 

0.05 

.10 

3.462 

6.398 

7.497 

11.58 

39.51 

33.81 

22.56 

6.387 

1.0000 

2.9191 


.25 

9.737 

13.63 

34.34 

2.331 

1.0157 

1.0654 


.50 

11.83 

14.38 

34.70 

2.219 

1.0263 

1.0142 


.75 

13.14 

14.70 

34.95 

2.191 

1.0337 

1.0014 


1.00 

15.00 

15.00 

35.00 

2.188 

1.0352 

1.0000 

3 

0.05 

.10 

.25 

2.677 

5.104 

8.471 

9.294 

12.23 

13.74 

41.07 

35.46 

34.87 

15.24 

4.494 

2.353 

1.0000 

1.0486 


.50 

10.95 

14.36 

35.46 

2.283 

1.0169 

1.0174 


.75 

12.72 

14.70 

35.83 

2.247 

1.0275 

1.0013 


1.00 

15.00 

15.00 

35.91 

2.244 

1.0298 

1.0000 

4 

0.05 

.10 

.25 

2.235 

4.360 

7.685 

10.33 

12.48 

13.74 

42.01 

36.68 

35.37 

10.96 

3.799 

2.415 

1.0000 

1.0495 


.50 

10.44 

14.36 

36.22 

2.335 

1.0240 

1.0148 


.75 

12.46 

14.70 

36.66 

2.305 

1.0365 

1.0017 


1.00 

15.00 

15.00 

36.81 

2.301 

1.0407 

1.0000 

5 

0.05 

.10 

.25 

2.020 

3.931 

7.232 

11.03 

12.71 

13.74 

43.62 

37.64 

36.03 

8.328 

3.429 

2.476 

1.0000 

1.0527 


.50 

10.09 

14.36 

36.92 

2.385 

1.0247 

1.0140 


.75 

12.34 

14.71 

37.48 

2.356 

1.0402 

1.0017 


1.00 

15.00 

15.00 

37.64 

2.352 

1.0447 

1.0000 

6 

0.05 

.10 

.25 

1.881 

3.653 

6.885 

11.52 

12.82 

13.75 

44.76 

39.00 

36.63 

6.195 

3.139 

2.515 

1.0000 

1.0466 


.50 

9.871 

14.36 

37.77 

2.431 

1.0311 

1.0117 


.75 

12.21 

14.69 

38.43 

2.407 

1.0491 

1.0017 


1.00 

15.00 

15.00 

38.46 

2.403 

1.0500 

1.0000 
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TABLE 1. - Concluded. 


(e) Comparison of F-only results for Cae,mx('^) with multiparameter results 



no 


1 

2 

3 

4 

5 

6 


V 


64.3 

4.0 

64.3 

4.0 

64.3 

4.0 

64.3 

■ 4.0 

64.3 

4.0 

64.3 

4.0 

a 

31.78 

2.180 

33.14 

2.252 

33.29 

2.301 

33.89 

2.309 

33.97 

2.394 

34.72 

2.408 

b 

38.09 

21.60 

39.51 

22.56 

41.07 

15.24 

42.01 

10.96 

43.62 

8.328 

44.76 

6.195 

c 

19.8 

890.8 

19.2 

901.8 

23.4 

562.3 

24.0 

374.7 

28.4 

247.9 

28.9 

157.3 

d 

33.63 

2.166 

34.70 

2.219 

35.46 

2.283 

36.22 

2.335 

36.92 

2.385 

37.77 

2.431 

e 

1 

5.8 1 
1 

-.6 

4.7 

-1.5 

6.5 

-.8 

6.9 

1.1 

8.7 

-.4 

8.8 

1.0 


3Cae,rnx niulti parameter strategy. 

*^Cae*mx F-only strategy, a = 0.05. 

^Percent increase of F-only (0.05) over multiparameter. 
*^Cae,mx F-only strategy, a = 0.50. 

^Percent increase of F-only (0.50) over multiparameter. 



TABLE 2. - FIRST FACTORIAL EXPERIMENT, ONE-HALF 
REPLICATE OF 2^ 

[Contents of Ni, W, and Zr held constant at 38, 14, 
and 0.25 wt. respectively; balance, Co.] 


(a) Levels of independent variables in natural units 


Vari able 

Design 

center 

Lower 

level 

Upper 

level 

Scale 

factor 


Level 

, Vlt. % 


^Ti 

1.0 

0.5 

1.5 

0.5 

^Cr 

4.0 1 

2.0 

6.0 

2.0 

^C 

.4 

.3 

.5 

.1 

^A1 

7.0 

6.75 

7.25 

.25 


— 

— 

— 

— 

C 

1593 

1566 

1621 

28 

(‘F) 

(2900) 

(2850) 

(2950) 

(50) 

1 


(b) Levels of independent variables in design units and corresponding results 


Alloy 

Level 

Stress-rupture® 

life, 

hr 

Elongation®, 

% 

^i 

Xp 

Cr 

"c 

^A1 


1 

-1 

-1 

-1 

-1 

-1 

175.1, 

199.4 

31, 

15 

2 

+1 

-1 

-1 

-1 

+1 

83.2, 

166.5 

4, 

8 

3 

-1 

+1 

-1 

-1 

+1 

22.9, 

24.5 

9, 

8 

4 

+1 

+1 

-1 

-1 

-1 

14.7, 

21.1 

3, 

8 

5 

-1 

-1 

+1 

-1 

+1 

153.5, 

237.6 

13, 

22 

6 

+1 

-1 

+1 

-1 

-1 

119.5, 

129.6 

10, 

10 

7 

-1 

+1 

+1 

-1 

-1 

28.2, 

39.0 

17, 

14 

8 

+1 

+1 

+1 

-1 

+1 

30.0, 

38.1 

14, 

11 

9 

-1 


-1 

+1 

+1 

55.1, 

79.2 

11, 

10 

10 



-1 

+1 

-1 

29.2, 

47.0 

5, 

8 

11 



-1 

+1 

-1 

3.5, 

11.1 

10, 

15 

12 



-1 

+1 

+1 

17.7, 

19.6 

14, 

13 

13 

-1 

-1 

n 

+1 

-1 

132.1, 

190.7 

19, 

16 

14 

+1 

-1 

19 

+1 

+1 

94.1, 

95.1 

10, 

9 

15 

-1 


■9 

+1 

+1 

! 12.7, 

19.1 

21, 

10 

16 

+1 

L—, 


mm 

+1 

-1 

16.7, 

16.8 

18, 

15 


^At stress of 103 MN/m^ (15 ksi) and temperature of 1010“ C (1850* F). 
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TABLE 3. - MODEL COEFFICIENT ESTIMATES FROM FIRST 
FACTORIAL EXPERIMENT 


Term 

Minimum 

C (4.0) 

ae,mx' 

Security 

regret 

Minimum 

C (64.3) 

ae,mx' 

Model coefficient estimates 

xO 

1.652 

1.652 

1.652 


-.030 

-.030 

.000 

Kr 

-.383 

-.383 

-.383 

XTiXCr 

.078 

.078 

.078 

K 

.100 

.100 

.100 

XTi^C 

-.017 

-.017 

.000 

xCrXC 

-.003 

-.003 

.000 

XAl^T 

.022 

.022 

.000 

XAl 

-.146 

-.146 

-.146 

XTiXAl 

.034 

.034 

.000 

xCrXAl 

-.002 

-.002 

.000 

XCXT 

-.045 

-.045 

.000 

XCXAI 

.042 

.042 

.000 

xcrXT 

.036 

.036 

.000 

XTiXT 

.052 

.052 

.052 

XT 

.037 

.037 1 

.000 

ndf 

1 

0 

2 

14 

s 

.000 i 

1 1 

.007 

.064 
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TABLE 4. - SECOND FACTORIAL EXPERIMENT, FULL 
REPLICATE OF 2^ 

[Contents of Ni, W, and Zr held constant at 38, 14, 
and 0.25 wt. ^ , respectively; balance, Co.] 


(a) Levels of independent variables in natural units 


Variable 

Design 

center 

Lower 

level 

Upper 

level 

Scale 

factor 


Level , 

wt. % 


^Ti 

1.0 

0.5 

1.5 

0.5 

^Cr 

2.0 i 

1.5 

2.5 

.5 

^C 

.5 

.4 

.6 

.1 

^A1 

6.75 

6.5 

7.0 

.25 


(b) Levels of independent variables in design units and corresponding results 


Alloy 

Level 

Stress-rupture^ 

life, 

hr 

Elongation^, 

% 

^Ti 

X 

o 

"c 

^A1 

1 

-1 

-1 

-1 

-1 

126.7, 

176.5 

13, 

12 

2 

+1 

-1 

-1 

-1 

196.0, 

184.1 

10, 

8 

3 

-1 

+1 

-1 

-1 

163.4, 

152.6 

24, 

12 

4 

+1 

+1 

-1 

-1 

194.0, 

249.4 

6, 

10 

5 

-1 

-1 

+1 

-1 

88.9, 

106.1 

15, 

25 

6 

+1 

-1 

+1 

-1 

175.8, 

160.1 

13, 

15 

7 

-1 

+1 

+1 

-1 

154.9, 

182.2 

17, 

14 

8 

+1 

+1 

+1 

-1 

144.1, 

162.4 

14, 

13 

9 

-1 

-1 

-1 

+1 

136.1, 

107.0 

17, 

14 

10 

+1 

-1 

-1 

+1 

65.7, 

60.0 

10, 

8 

11 

-1 

+1 

-1 

+1 

129.8, 

107.2 

11, 

15 

12 

+1 

+1 

-1 

+1 

80.6, 

87.7 

9, 

11 

13 

-1 

-1 

+1 

+1 

175.8, 

164.8 

19, 

15 

14 

+1 

-1 

+1 

+1 

167.2, 

166.1 

13, 

12 

15 

-1 

+1 

+1 

+1 

141.8, 

129.2 

14, 

19 

16 

+1 

+1 

+1 

+1 

145.0, 

140.1 

13, 

12 


^At stress of 103 MN/m^ (15 ksi) and temperature of 1010° C (1850° F). 
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TABLE 4. - Concluded. 


(c) Results from three deletion strategies 



Minimum Cgg, 

mx ( ^ 0 ) 

Security regret 

Minimum Cae,mx(®^*^) 


A 

P 

ndf 

s 

A 

P 

ndf 

s 

A 

P 

ndf 

s 

0 

15 

0 

0.0 

15 

1 

0.007 

4 

15 

0.036 

1 

8 

13 

.025 

4 

16 

.035 

4 

16 

.035 

2 

14 

3 

.066 

4 

15 

.032 

4 

16 

.036 

3 

14 

4 

.049 

6 

14 

.025 

4 

17 

.034 

4 

5 

15 

.024 

96 

915 

9.024 

! 

4 

18 

.032 


^See illustrative output, appendix E. 
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(e) Hq = 5. (f) Oq = 6. 

Figure 1. - Concluded, 


I PPOJtn NUMBER 


FORTRAN STATEMENT 


,5^cc 

'i. 

X 

,4, 

,5, 

. 6 . 

7. 

.8, 

<9, 

10 . 

, 1 , 1 . 

, 1,2 
U 
14 
15. 
. 1 . 6 . 
C’l. 
,Cr.2. 
,C.-3, 
C-,4, 
.0 
0 
0 


lb JA.CTQR I AL EX PE RjM 


21.2^ 
21,96 
21,6? 
,2l9.4 
2 8^ 
21.75 
,2154 
,2144 
2156 
2 .65 
2129 
2 .8p 
2175 
21,67 
21,41 
,21,45 
,21,95 
,21,14 
,214,9 
21.58 
. ,90 
. 90 


176 

184 

1.52 

24.9 

10,6 

1,60 

1.82 

1.62 

1,07 

,60 

1,07 

.87 

164. 

16,6 

129 

140 

|l,8.5 

8,8 

1.1,9, 

14,3 


,2 
1 
6. 

0 , 

8 , 

2 

85 

8,0 

_^i0. 

I 20 21 22 2i ?(*' 
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